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In  this  paper,  we  discuss  the  validation  of  water  level  and  current  predictions  from 
three  coastal  hydrodynamic  models  and  document  the  resource  and  operational  re¬ 
quirements  for  each  modeling  system.  The  ADvanced  CIRCulation  Model  (ADCIRC), 
the  Navy  Coastal  Ocean  Model  (NCOM),  and  DelftdD  have  been  configured  and  val¬ 
idated  for  the  Chesapeake  Bay  region  during  a  Navy  exercise.  Water  level  predictions 
are  compared  with  a  NOAA/NOS  water  level  gauge  at  the  Chesapeake  Bay  Bridge 
Tunnel  location  while  current  predictions  are  validated  with  Acoustic  Doppler  Profiler 
(ADP)  measurement  records  at  three  locations  in  the  lower  Chesapeake  Bay.  Statis¬ 
tical  metrics  such  as  correlation  coefficient  and  root  mean  square  error  (RMSE)  are 
computed.  Both  the  vertically-integrated  currents  and  currents  at  varying  water  depths 
are  compared  as  well.  The  model-data  comparisons  for  surface  elevation  indicate  all 
three  models  agreed  well  with  water  level  gauge  data.  The  two-dimensional  version  of 
ADCIRC,  ADCIRC2D,  and  NCOM  yield  better  statistics,  in  terms  of  correlation  and 
RMSE,  than  DelftSD.  For  vertically-integrated  currents,  ADCIRC2D  has  the  smallest 
RMSE  at  Thimble  Shoal  and  Naval  Station  locations  while  NCOM  has  the  smallest 
RMSE  at  Cape  Henry.  For  the  horizontal  currents  over  the  water  column,  the  fully 
three-dimensional,  baroclinic  ADCIRC  model,  ADCIRC3D,  and  NCOM  both  showed 
better  agreement  with  the  ADP  measurements. 
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1.  Introduction 

There  is  a  strong  need  for  the  U.S.  Navy  to  develop  relocatable,  operational  coastal  forecast 
systems  to  support  naval  missions  in  coastal  and  semi-enclosed  seas.  Naval  Research 
Laboratory  (NRL)  has  been  actively  working  on  the  development  of  multiple  global  and 
regional  ocean  models  for  that  purpose  (Chu  et  al.  2009).  Navy  Coastal  Ocean  Model 
(NCOM),  ADvanced  CIRCulation  Model  (ADCIRC),  and  DELFT3D  are  some  of  those 
models.  Products  of  those  models  such  as  water  levels,  currents  and  temperature  are  used 
to  support  fleet  navigation,  Mine  Warfare  (MIW),  diver  operations,  and  so  forth. 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

2q  J2  2-  REpORT  TYpE 

4.  TITLE  AND  SUBTITLE 

Multi-Model  Validation  of  Currents  in  the  Chesapeake  Bay  Region  in 
June  2010 

6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

Naval  Research  Laboratory, Stennis  Space  Center, MS, 39529-5004 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 

18.  NUMBER 

19a.  NAME  OF 

ABSTRACT 

OF  PAGES 

RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

30 

3.  DATES  COVERED 

00-00-2012  to  00-00-2012 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Downloaded  by  [Naval  Research  Laboratory],  [Philip  CHU]  at  07:54  04  December  2012 


400 


P.-Y.  Chu  et  al. 


Figure  1.  The  U.S.  map  and  Chesapeake  Bay  location.  (Color  figure  available  online.) 


A  Navy  exercise  in  the  lower  Chesapeake  Bay  region  during  June  2010  provided  an 
excellent  opportunity  to  validate  the  accuracy  and  performance  of  these  models.  Chesapeake 
Bay  lies  on  the  eastern  coast  of  the  United  States  (Figure  1).  It  is  the  largest  inlet  along  the 
U.S.  Atlantic  Coastal  Plain  and  is  the  largest  estuary  in  the  United  States.  It  lies  off  of  the 
Atlantic  Ocean  and  is  surrounded  by  the  states  of  Virginia  and  Maryland.  The  study  area 
focuses  on  the  region  surrounding  the  U.S.  Naval  Station,  Norfolk,  Va.,  slightly  more  than 
33  km  west  of  mouth  of  the  bay.  The  primary  bathymetric  feature  of  this  area  is  Thimble 
Shoal  Channel.  The  NOAA  station  at  Thimble  Shoal  and  the  seaward  extension  of  the 
channel  are  apparent  in  the  bathymetric  contours  seen  in  Figure  13. 

In  this  paper,  we  describe  (1)  the  modeling  effort  by  NRL  scientists  to  support  the 
exercise;  (2)  the  validation  and  performance  of  water  level  and  current  predictions  by 
three  coastal  hydrodynamic  models:  the  ADCIRC,  both  in  two-dimensional  (ADCIRC2D) 
and  three-dimensional,  baroclinic  (ADCIRC3D)  forms,  the  NCOM  and  Delft3D;  and  (3) 
the  resource  requirements  including  hardware,  personnel,  training  and  operations  for  each 
modeling  system.  This  paper  is  organized  as  follows:  Section  2  describes  model  config¬ 
uration  and  products.  Observational  and  field  data  are  summarized  in  Section  3.  Model 
validation  and  skill  assessment  are  detailed  in  Section  4.  Resource  issues  and  require¬ 
ments  for  the  modeling  systems  are  discussed  in  Section  5.  Conclusions  are  summarized  in 
Section  6. 


2.  Model  Configuration  and  Products 

ADCIRC  is  a  finite  element-based  community  coastal  circulation  model  that  solves  water 
surface  elevation  using  the  continuity  equation  in  the  Generalized  Wave-Continuity  Equa¬ 
tion  (GWCE)  form  and  solves  velocity  using  the  momentum  equations.  Its  unstructured 
grid  and  unique  wetting/drying  feature  allows  accurate  modeling  of  complex  coastlines  and 
estuaries  at  fine  spatial  scales.  This  model  can  be  run  either  in  its  two-dimensional  (2D) 
depth-integrated  mode  or  in  a  full  three-dimensional,  baroclinic  mode.  The  detailed  for¬ 
mulation  and  implementation  of  ADCIRC  can  be  found  in  Luettich  and  Westerink  (2004, 
2005)  and  a  recently  published  report  by  Blain  et  al.  (2010).  The  two  versions,  ADCIRC2D 
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and  ADCIRC3D,  implemented  for  this  validation  exercise  and  their  key  differences  are 
summarized  below: 

1)  ADCIRC2D,  the  2D  depth-integrated  code,  is  based  herein  on  version  45. 1 1  while 
ADCIRC3D,  the  fully  three-dimensional  baroclinic  code  as  applied  within  is  based 
on  version  49.00. 

2)  ADCIRC2D  computes  a  two-dimensional  vertically-integrated  velocity  while  AD- 
CIRC3D  computes  a  three-dimensional  velocity  field  with  41  layers  in  the  vertical 
direction  using  a  generalized  sigma  coordinate  system. 

3)  The  mesh  over  which  ADCIRC2D  is  applied  has  spatial  resolution  ranging  from 
15  m  to  2  km;  application  of  the  ADCIRC3D  model  uses  a  coarser  resolution  mesh 
whose  minimum  element  spacing  is  approximately  150  m. 

4)  Both  ADCIRC2D  and  ADCIRC3D  applies  a  tidal  database  as  the  open  ocean 
boundary  condition,  but  ADCIRC3D  may  also  derive  boundary  and  initial  condi¬ 
tions  from  the  U.S.  East  Coast  NCOM  forecasts. 

5)  ADCIRC3D  has  additional  forcing  in  the  form  of  surface  wind  and  heat  flux 
obtained  from  the  Navy  Coupled  Ocean  Atmosphere  Mesoscale  Prediction  System 
(COAMPS)  (Hodur  1997)  at  its  operational  resolution  of  27  km. 


2.7.  ADCIRC2D 

The  ADCIRC2D  Chesapeake  Bay  model  system  was  configured  to  cover  the  Chesapeake 
Bay,  Delaware  Bay  and  extends  east  to  the  Atlantic  Ocean  (73W-77W,  36N-40N).  The  mesh 
contained  318,860  nodes  and  558,718  elements  with  15  m  spatial  resolution  in  the  lower 
Chesapeake  Bay  and  shipping  channels  and  approximately  2  km  at  the  outer  boundary. 
The  grid  bathymetry  was  derived  from  a  combination  of  NOAA/NOS  soundings,  NOAA 
Electronic  Navigation  Charts  (ENCs),  and  NOAA  Raster  Nautical  Charts  (RNCs).  The 
tidal  potential  and  tidal  constituents  applied  at  the  open  ocean  boundary  were  extracted 
from  a  tidal  database  derived  from  the  Western  North  Atlantic  Ocean  Tidal  Model  (Yang 
and  Myers  2007);  eight  main  tidal  constituents  were  included:  Ql,  01,  Kl,  N2,  M2,  K2, 
S2,  and  PI.  River  discharge  was  determined  to  be  negligible  during  the  validation  period 
and  was  therefore  neglected.  The  bathymetry  and  numerical  meshes  for  the  computation 
domain  are  depicted,  respectively,  in  Figures  2  and  3.  The  validation  region  in  the  lower 
Chesapeake  Bay  is  shown  in  Figure  4. 

The  ADCIRC2D  model  system  was  configured  in  March  2010.  After  a  brief  spin-up 
and  hindcast  validation  using  historical  records,  daily  predictions  of  water  level  and  currents 
were  produced  starting  in  April  2010  and  continued  to  run  in  real-time  for  the  duration  of 
the  exercise  period  (June  4-11,  2010).  In  this  study,  ADCIRC  is  run  in  a  parallel  fashion  on 
a  cluster  of  computers.  The  parallel  environment  allows  the  use  of  multiple  interconnected 
processors  simultaneously  to  decrease  runtimes.  The  model  forecasts  used  a  time  step  of 
1  second  and  ran  over  64  CPUs;  at  NRL,  the  daily  forecast  was  executed  on  a  Linux 
cluster  using  the  Sun  Grid  Engine  (SGE)  queue  system,  on  which  a  72-hr  forecast  took 
approximately  1  hr.  Identical  runs  were  performed  on  the  DoD  Supercomputing  Resources 
Center  (DSRC)  host  “DaVinci”  at  the  Naval  Oceanographic  Office  (NAVOCEANO).  The 
same  model  configuration  also  took  approximately  1  hr  of  wall  clock  time  for  a  72-hr 
forecast.  The  daily  products  for  the  system  included  hourly  two-dimensional  maps  of  water 
levels  and  currents  in  the  lower  Chesapeake  Bay  (e.g.,  Figures  5  and  6).  In  addition,  6-min 
water  level  and  current  magnitude  time  series  at  ten  locations  were  generated  daily  to 
support  the  exercise.  Examples  of  these  products  are  shown  in  Figures  7  and  8. 
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Figure  2.  The  bathymetry  for  the  ADCIRC2D  domain.  (Color  figure  available  online.) 


2.2.  ADCIRC3D 

As  described  earlier,  ADCIRC3D  is  the  full  three-dimensional  (3D)  baroclinic  version  of 
ADCIRC.  This  version  of  ADCIRC  solves  the  transport  equations  for  temperature  and 
salinity  using  a  terrain  following  generalized  sigma  vertical  coordinate  system  in  which 
the  nodes  can  be  distributed  over  the  vertical  direction.  The  stretched  coordinate  system 
is  applied  to  all  terms  except  the  baroclinic  pressure  gradient  to  reduce  known  limitations 
when  using  sigma  coordinates. 

The  ADCIRC  3D  domain  has  the  same  geographic  coverage  as  the  ADCIRC2D  model 
but  uses  an  unstructured  mesh  with  99,309  nodes,  192,051  elements  leading  to  coarser 
resolution  of  about  150  m  in  the  lower  Chesapeake  Bay;  41  uniformly  distributed  sigma 
layers  are  used  over  the  vertical.  The  bathymetry  of  this  mesh  was  interpolated  from  same 
sources  as  the  ADCIRC  2D  model.  Boundary  and  initial  conditions  were  derived  from  the 
U.S.  East  Coast  NCOM  forecasts.  COAMPS  27-km  winds  at  3-hr  intervals  were  applied  as 
surface  meteorological  forcing.  The  surface  heat  fluxes  are  calculated  using  latent,  sensible 
heat  fluxes,  and  shortwave  and  longwave  radiation  components  provided  by  COAMPS. 
Similar  to  ADCIRC2D,  river  discharge  was  not  included  as  boundary  forcing. 

ADCIRC3D  runs  start  with  a  diagnostic  phase  during  which  the  temperature,  salinity, 
and  density  fields  are  unchanged.  This  is  intended  to  spin-up  the  winds,  tides,  and  other 
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Figure  3.  The  unstructured  mesh  used  for  ADCIRC2D  simulations. 


Figure  4.  The  unstructured  mesh  of  the  ADCIRC2D  model  in  the  lower  Chesapeake  Bay.  (Color 
figure  available  online.) 
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Elcvation(m)  on  06/07/2010  at  20:00  (m) 


Figure  5.  The  water  surface  elevation  result  from  ADCIRC2D  on  June  7,  2010.  (Color  figure 
available  online.) 


barotropic  forcing.  The  diagnostic  run  is  followed  by  a  prognostic  run  in  which  full  3D 
baroclinic  calculations  are  performed  and  the  transport  equations  for  temperature  and 
salinity  are  solved  producing  density-driven  currents.  The  Mellor-Yamada  2.5  turbulent 
closure  scheme  is  selected  as  the  vertical  mixing  scheme. 

The  ADCIRC3D  system  was  configured  for  the  Chesapeake  Bay  region  using  a  time 
step  of  5  s  and  again  executing  on  64  CPUs.  The  same  SGE  parallel  computing  cluster  at 
NRL  as  was  used  for  the  ADCIRC2D  application  was  used  to  make  72-hr  diagnostic  runs 
followed  by  72-hr  prognostic  run  simulations.  Because  of  the  computational  requirements 
and  the  need  to  wait  for  the  completion  of  NCOM  forecasts  that  are  applied  as  boundary 


Velocity  (m/s)  on  06/07/2010  at  20:00 


Figure  6.  The  depth-integrated  current  result  from  ADCIRC2D  on  June  7,  2010.  Color  is  current 
magnitude  and  direction  is  shown  by  black  arrows.  (Color  figure  available  online.) 
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Eleva[bn(m)  ai  CBBT  on  06/06/2010 


Figure  7.  The  water  surface  elevation  results  from  ADCIRC2D  from  June  6-8,  2010,  at  the  Chesa¬ 
peake  Bay  Bridge  Tunnel. 


conditions,  the  ADCIRC3D  runs  were  performed  in  a  non-real  time  delayed  mode.  The 
simulations  were  run  daily  on  64  CPUs  and  required  approximately  4  hrs  of  wall-clock 
time.  The  performance  of  the  NRL  SGE  has  been  comparable  to  the  performance  Navy 
DSRC  host  DaVinci.  Products  of  water  level  and  horizontal  current  maps  as  well  as  station 
time  series,  identical  to  those  produced  by  the  ADCIRC2D  system,  are  also  produced  by 
this  system.  Additional  products  include  temperature,  salinity,  and  horizontal  current  fields 
over  the  water  column  at  6-hr  intervals. 


Figure  8.  The  depth-integrated  current  result  from  ADCIRC2D  from  June  6-8, 2010,  at  Chesapeake 
Bay  Bridge  Tunnel.  (Color  figure  available  online.) 
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2.3 .  NCOM 

The  Navy  Coastal  Ocean  Model  (NCOM)  is  a  baroclinic,  hydrostatic  with  Boussinesq 
approximation,  free  surface,  data  assimilated  model  developed  by  NRL.  NCOM  uses  a 
Cartesian  horizontal  grid  system,  a  flexible  hybrid  sigma-z  in  the  vertical  coordinate,  an 
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Figure  9.  The  Chesapeake  Bay  NCOM  domain.  (Color  figure  available  online.) 


Downloaded  by  [Naval  Research  Laboratory],  [Philip  CHU]  at  07:54  04  December  2012 


Validation  of  Currents  in  the  Chesapeake  Bay  Region 


407 


implicit  scheme  for  free  surface,  and  Mellor-Yamada  level  2  closure  for  the  vertical  mixing. 
Complete  descriptions  of  the  model  formulation  and  implementation  can  be  found  in 
Martin  (2000)  and  Barron  et  al.  (2006).  NCOM  has  been  transitioned  to  the  NAVOCEANO 
Operational  Production  Center  to  provide  daily  ocean  forecasts  to  the  U.S.  Navy  at  global, 
regional  and  coastal  scales  (Rowley  2008,  2010). 

The  NCOM  model  in  the  Chesapeake  Bay  region  for  this  exercise  is  configured  in 
the  following  fashion:  The  domain  is  a  5-by-5  degree  area  (72.5W-77.5W,  34.5N-39.5N) 
that  covers  the  Chesapeake  Bay  and  part  of  the  U.S.  East  Coast  (Figure  9)  at  500  m 
spatial  resolution  with  29  vertical  layers.  The  computational  domain  included  more  than 
one  million  grid  points.  Bathymetry  was  derived  from  the  NRL  DBDB2  global  bathymetry 
database.  Boundary  forcing  and  initial  conditions  were  extracted  from  the  East  Coast 
NCOM  which  has  a  3  km  grid  resolution.  Surface  meteorological  forcing  was  applied 
using  the  COAMPS  forecast  meteorological  fields. 

The  NCOM  simulations  were  run  daily  on  128  CPUs  at  the  Navy  DSRC  host  DaVinci 
and  required  approximately  5  hrs  of  wall-clock  time  for  72-hr  forecasts,  including  data 
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Figure  10.  The  Chesapeake  Bay  Delft3D  domain.  (Color  figure  available  online.) 
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assimilation  and  post-processing.  In  addition  to  the  standard  water  level  and  current  fore¬ 
casts,  NCOM  also  generated  three  dimensional  temperature  and  salinity  fields  at  3-hr 
intervals. 


2.4.  Delft3D 

The  Delft3D  modeling  system,  developed  by  Delft  Hydraulics  (www.deltares.nl),  is  capa¬ 
ble  of  simulating  hydrodynamic  processes  due  to  wind,  tides,  and  waves  for  coastal  and 
estuarine  areas.  The  model  can  be  run  in  2D  or  3D  configuration.  A  GUI-based  preprocess¬ 
ing  tool  is  used  to  generate  curvilinear  or  rectangular  grids  in  Cartesian  coordinates  and 
the  post-processing  tool  allows  production  of  graphics  and  plotting  from  the  native  binary 
model  output  format  (Deltares  2011).  Delft3D  can  be  run  on  either  a  personal  computer 
(PC)  Windows  or  Linux  platform;  however,  parallel  processing  capability  is  not  currently 
implemented. 

The  system  was  configured  with  a  curvilinear  grid  with  approximately  500  cells  in 
both  x  and  y  coordinates  at  183  m  spatial  resolution  (Figure  10)  and  4  layers  in  the  vertical 
direction  distributed  at  the  surface,  20%,  60%,  and  100%  of  the  total  depth.  Boundary 
conditions  also  came  from  the  East  Coast  NCOM  model.  The  72-hour  forecast  Delft3D 
model  simulation  was  performed  on  a  single  processor  PC.  The  wall  clock  time  for  a  single 
forecast  run  averaged  approximately  5  hr.  An  example  plot  showing  predicted  water  level 
for  0000  hr  on  June  14,  2010,  is  shown  in  Figure  11. 

Table  1  summarizes  configuration  parameters  for  all  of  the  models  described  herein. 
Spatial  resolution,  number  of  vertical  layers,  number  of  grid  nodes,  open  boundary  condi¬ 
tions  (OBCs),  surface  forcing,  and  parallel  operation  are  included. 
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Figure  11.  Example  of  Delft3D  water  level  prediction.  (Color  figure  available  online.) 
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Table  1 


Model  configuration  summary 


ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

Spatial  resolution 

15  m-2  km 

150  m-12  km 

500  m 

183  m 

Vertical  layers 

NA 

41 

29 

4 

No.  of  grid  nodes 

320  K 

100  K 

1  million  (1000  x 

250  K  (500  x 

unstructured 

unstructured 

1000) 

500) 

OBC  and  surface 

N.  Atlantic  Tidal 

EC-NCOM 

EC-NCOM 

EC-NCOM 

forcing 

Database 

COAMPS 

COAMPS 

COAMPS 

Parallel 

Yes 

Yes 

Yes 

No 

environment 

3.  Observational  Field  Data 

3.1.  Meteorological  Conditions 

Several  severe  storms  passed  the  Chesapeake  Bay  region  during  the  June  2010  exercise 
period,  providing  excellent  opportunity  for  the  model-data  comparison.  Figure  12  shows 
the  wind  speed,  gusts  and  directions  at  the  Chesapeake  Bay  Bridge  Tunnel  (CBBT)  lo¬ 
cation  during  the  exercise  period.  There  are  at  least  three  occasions  where  wind  speed 
exceeded  12  m/s  and  a  total  direction  change  (360  degrees)  within  just  a  few  hour 
periods. 


3.2.  Water  Level  Data 

For  the  water  level  analyses,  validation  data  were  obtained  from  NOAA/NOS  water  level 
gauge  at  CBBT  (NOAA  Station  ID:  CBBV2-8638863).  The  data  at  CBBT  are  recorded  in 


N0AA/N0S/C0-0PS 
Wind  Speed/Gusts/Dir 

8638863  Chesapeake  Bay  Bridge  Tunnel,  VA 
from  2010/06/04  -  2010/06/12 


T3 
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Figure  12.  Wind  speed,  gusts  and  directions  at  CBBT  location  on  June  4-12,  2010.  (Color  figure 
available  online.) 
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6-min  intervals.  Data  during  the  exercise  period  from  the  location  were  used  for  water  level 
validation. 


3.3.  Current  ADP  data 

Three  downward-looking  ADPs  at  locations  in  the  bay  were  used  to  collect  velocity  infor¬ 
mation  through  the  water  column.  Near  real-time  Acoustic  Doppler  Profiler  (ADP)  records 
collected  by  NOAA/NOS  at  Cape  Henry  (NOAA  Station  ID:  CB0102),  Thimble  Shoal 
(NOAA  Station  ID:  CB0301)  and  Naval  Station  (NOAA  Station  ID:  CB0402)  locations 
(Figure  13)  were  used  to  validate  the  model  currents  during  the  exercise  period.  The  ADP 
bin  size  was  1  m  and  the  sampling  rate  was  6  min.  No  detiding  procedure  has  been  applied. 
Since  the  ADP  data  used  are  near  real-time,  they  have  not  been  post  processed  through  the 
standard  NOAA  Quality  Assurance/Quality  Control  (QA/QC)  procedure.  Industry  standard 
procedures  were  followed  to  identify,  gap-fill,  and  interpolate  missing  records. 


4.  Validation  Test  Results 

The  data  collected  by  NOAA/NOS  during  the  exercise  were  used  for  model  validation  and 
comparison.  Figure  13  shows  the  locations  of  water  level  gauge  at  CBBT  and  NOAA/NOS 
ADP  current  meters  at  Cape  Henry,  Thimble  Shoal,  and  Naval  Station. 


longitude 


Figure  13.  ADCIRC  bathymetry  with  validation  locations.  (Color  figure  available  online.) 


Downloaded  by  [Naval  Research  Laboratory],  [Philip  CHU]  at  07:54  04  December  2012 


Validation  of  Currents  in  the  Chesapeake  Bay  Region 


411 


4.1.  Water  Levels 

For  the  water  level  validation,  6-min  interval  water  level  data  collected  at  CBBT  are  used  for 
the  validation.  The  model-predicted  water  level  fluctuations  are  referenced  to  the  Mean  Sea 
Level  (MSL)  while  observational  tidal  gauges  are  generally  referenced  to  the  NOAA  mean 
lower  low  water  datum  (MLLW).  Adjustments  were  made  to  the  model  output  to  match 
the  tidal  vertical  datum  in  order  to  make  the  statistical  comparisons.  The  four  modeled 
water  level  time  series  from  ADCIRC2D,  ADCIRC3D,  NCOM  and  Delft3D  results  were 
de-trended  and  then  plotted  along  with  tidal  gauge  data  from  CBBT  in  Figure  14.  All 
four  models  performed  reasonably  well  for  water  level  prediction.  ADCIRC3D  tends  to 
overestimate  the  water  levels  because  the  daily  open  boundary  condition  provided  by 
NCOM  introduces  a  weak  tidal  signal  to  the  domain  in  addition  to  the  tidal  constituents 
provided  by  the  external  dial  database.  Blain  et  al.  (2012)  investigates  the  sources  of  error 
in  predicting  water  levels  and  reports  that  using  the  external  tidal  database  only  for  tidal 
forcing  leads  to  better  predictions.  Delft3D  results  showed  a  slight  phase  lag.  NCOM 
shows  a  phase  lead;  ADCIRC  has  good  phase  characteristics.  Table  2  shows  the  RMSE  and 
correlation  coefficient  with  respect  to  water  level  during  June  6-14,  2010,  for  each  of  the 
models. 

As  shown  in  Table  2,  all  four  models  predict  the  water  levels  at  the  Chesapeake 
Bay  Bridge  Tunnel  measurement  station  with  a  high  correlation  coefficient  (R  >  0.77). 
ADCIRC2D  predictions  for  the  water  levels  have  the  least  error  with  the  highest  correlation 


NCOM 
■■  DELFT 
-  Measurements 


06A)6/2010  06/07/2010  06/08/2010  06/09/2010  06/10/2010  06/11/2010  06/12/2010  06/13/2010  06/14/2010 

Figure  14.  The  water  surface  elevation  measured  by  Chesapeake  Bay  Bridge  Tunnel  tide  gauge  vs. 
model  results  between  June  6  and  14,  2010.  (Color  figure  available  online.) 
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Table  2 


Summary  of  water  level  statistics 


ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

Correlation  coefficient 

0.903 

0.865 

0.796 

0.773 

Root  Mean  Square  Error  (m) 

0.131 

0.161 

0.171 

0.183 

coefficient  of  0.9.  The  ADCIRC3D  follows  with  a  correlation  coefficient  of  0.87.  NCOM 
predicts  the  water  levels  with  a  correlation  coefficient  of  almost  0.8  and  Delft3D  produces 
water  level  predictions  with  the  least  correlation.  The  performance  of  the  models  in  predict¬ 
ing  the  water  levels  was  also  evaluated  using  the  RMSE.  Once  again  ADCIRC2D  performs 
the  best  and  results  in  the  least  error  while  Delft3D  produces  the  largest  error.  ADCIRC2D 
has  the  highest  resolution  over  the  shallow  coastal  waters,  which  is  expected  to  be  an  im¬ 
portant  reason  for  better  model  performance.  ADCIRC  has  been  used  for  numerous  coastal 
surge  and  inundation  studies  and  is  used  operationally  during  hurricane  season.  Because 
of  that,  the  model  has  been  developed  towards  getting  more  accurate  water  level  results 
and  the  performance  of  ADCIRC  for  water  level  predictions  may  have  been  expected  to  be 
better  than  NCOM  and  DELFT3D.  Although  NCOM  assimilates  water  level  data,  in  this 
application  NCOM  water  level  predictions  at  this  relatively  shallow  nearshore  location  are 
of  poorer  quality  than  those  of  ADCIRC. 
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Figure  15.  Depth-integrated  currents  measured  at  NO  A  A  current  meter  stations;  Cape  Henry,  Thim¬ 
ble  Shoal,  and  Naval  Station;  compared  with  the  model  results  from  ADCIRC2D,  ADCIRC3D, 
R-NCOM,  and  DELFT3D  on  June  4-6,  2010.  (Color  figure  available  online.) 
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Figure  16.  Depth-integrated  currents  measured  at  NOAA  current  meter  stations;  Cape  Henry,  Thim¬ 
ble  Shoal,  and  Naval  Station;  compared  with  the  model  results  from  ADCIRC2D,  ADCIRC3D, 
R-NCOM,  and  DELFT3D  on  June  6-8,  2010.  (Color  figure  available  online.) 


4.2.  Vertically -Integrated  Currents 

Near  real-time  NOA A/NOS  ADP  records  at  three  locations  have  been  used  to  validate  the 
model  currents  during  the  exercise  period.  Figures  15-19  show  the  depth-integrated  mea¬ 
sured  currents  and  the  depth-integrated  model  predictions  for  successive  two-day  periods 
at  Cape  Henry,  Thimble  Shoal,  and  Naval  Station.  ADCIRC2D  and  ADCIRC3D  directly 
output  the  depth-integrated  currents.  For  NCOM  and  Delft3D,  the  depth-varying  currents 
through  the  water  column  were  averaged  for  comparison.  The  measured  currents  exhibit 
semi-diurnal  variability  with  a  mean  magnitude  hovering  near  0.5  m/s;  peak  currents  reach 
1  m/s  on  only  a  couple  of  occasions  near  the  end  of  the  time  period,  June  10-14.  It  may  be 
seen  in  the  figures  that  all  four  models  did  fairly  well  in  predicting  the  current  magnitude. 
No  single  model  is  observed  to  stand  out  with  its  accuracy  and  performance  according 
to  these  qualitative  comparisons.  In  general,  all  four  models  seem  to  underestimate  the 
currents. 

Tables  3-5  and  Figure  20  show  the  correlation  coefficients  of  depth-integrated  currents 
for  all  four  models  at  the  Cape  Henry,  Thimble  Shoal,  and  Naval  Station  locations  during 
June  4-10,  2010.  The  statistics  were  calculated  for  each  48-hr  period  as  well  as  for  the 
whole  6-day  period.  NCOM  has  the  highest  correlation  coefficient  (0.745)  at  Cape  Henry 
while  the  other  models  perform  similar.  On  the  other  hand,  ADCIRC2D  shows  the  highest 
correlation  coefficients  at  Thimble  Shoal  and  Naval  Station.  A  higher  correlation  coefficient 
may  be  considered  to  indicate  less  phase  error. 
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Thimble  Shoal 


Figure  17.  Depth-integrated  currents  measured  at  NOAA  current  meter  stations;  Cape  Henry,  Thim¬ 
ble  Shoal,  and  Naval  Station;  compared  with  the  model  results  from  ADCIRC2D,  ADCIRC3D, 
R-NCOM,  and  DELFT3D  on  June  8-10,  2010.  (Color  figure  available  online.) 


Tables  6-8  and  Figure  21  show  the  root-mean-square  errors  (RMSE)  of  depth- 
integrated  currents  for  all  four  models  at  the  three  stations.  Overall,  NCOM  has  the  smallest 
error  over  the  6-day  period  at  Cape  Henry,  and  ADCIRC2D  has  the  smallest  error  over  the 
6-day  period  at  Thimble  Shoal  and  at  Naval  Station. 

NCOM  is  the  only  model  in  this  study  with  data  assimilation  that  is  used  to  improve 
model  performance.  As  a  result,  NCOM  produces  the  best  predictions  at  Cape  Henry, 
which  is  the  deepest  station  closer  to  the  Chesapeake  Bay  mouth.  ADCIRC  performs 
better  at  shallower  stations  probably  because  of  the  higher  grid  resolution  at  shallower 
locations.  ADCIRC2D  predicts  depth  averaged  currents  more  accurately  than  ADCIRC  3D. 
The  winds  used  in  ADCIRC3D  are  a  major  source  of  error  for  current  predictions  while  the 


Table  3 


Correlation  coefficients  at  Cape  Henry  for  depth-integrated  currents 


ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

6/4— 6/6 

0.513 

0.381 

0.485 

0.220 

6/6— 6/8 

0.494 

0.545 

0.706 

0.539 

6/8-6/10 

0.375 

0.469 

0.824 

0.466 

6/4-6/10 

0.439 

0.455 

0.745 

0.463 
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Figure  18.  Depth-integrated  currents  measured  at  NOAA  current  meter  stations;  Cape  Henry,  Thim¬ 
ble  Shoal,  and  Naval  Station;  compared  with  the  model  results  from  ADCIRC2D,  ADCIRC3D, 
R-NCOM,  and  DELFT3D  on  June  10-12,  2010.  (Color  figure  available  online.) 


Table  4 

Correlation  coefficient  at  Thimble  Shoal  for  depth-integrated  currents 


ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

6/4-6/6 

0.168 

0.237 

0.024 

-0.370 

6/6— 6/8 

0.390 

0.429 

0.328 

0.182 

6/8-6/10 

0.597 

0.223 

0.555 

0.590 

6/4-6/10 

0.491 

0.299 

0.442 

0.343 

Table  5 

Correlation  coefficient  at  Naval  Station  for  depth-integrated  currents 

ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

6/4-6/6 

0.365 

-0.270 

0.184 

-0.024 

6/6— 6/8 

0.271 

0.021 

0.005 

0.073 

6/8-6/10 

0.419 

0.084 

0.067 

0.056 

6/4-6/10 

0.344 

0.027 

0.083 

0.039 
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Figure  19.  Depth-integrated  currents  measured  at  NOAA  current  meter  stations;  Cape  Henry,  Thim¬ 
ble  Shoal,  and  Naval  Station;  compared  with  the  model  results  from  ADCIRC2D,  ADCIRC3D, 
R-NCOM,  and  DELFT3D  on  June  12-14,  2010.  (Color  figure  available  online.) 


Table  6 

RMSE  at  Cape  Henry  for  depth-integrated  currents 


ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

6/4—676 

0.145 

0.171 

0.157 

0.189 

6/6— 6/8 

0.264 

0.256 

0.194 

0.254 

6/8-6/10 

0.285 

0.280 

0.170 

0.277 

6/4-6/10 

0.239 

0.241 

0.174 

0.243 

Table  7 

RMSE  at  Thimble  Shoal  for  depth-integrated  currents 

ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

6/4— 6/6 

0.108 

0.151 

0.123 

0.129 

6/6— 6/8 

0.117 

0.141 

0.136 

0.141 

6/8-6/10 

0.123 

0.183 

0.139 

0.165 

6/4-6/10 

0.116 

0.159 

0.133 

0.146 
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Figure  20.  The  correlation  coefficients  of  numerical  models  for  depth-integrated  currents  predictions 
between  June  4  and  10,  2010,  at  Cape  Henry,  Thimble  Shoal,  and  Naval  Station  locations.  (Color 
figure  available  online.) 


initialization  of  the  ADCIRC3D  by  NCOM  predictions  may  also  decrease  accuracy  (Blain 
et  al.  2012). 


4.3.  Vertical  Variation  of  Currents 

The  horizontal  currents  over  the  water  column  of  all  three  3 -dimensional  models  are 
compared  with  the  NOAA  ADP  instruments  at  Cape  Henry,  Thimble  Shoal,  and  Naval 
Station.  Figures  22-26  show  the  currents  at  different  times  during  the  exercise.  NCOM 
produces  output  at  7  (at  Cape  Henry  and  Naval  Station)  or  8  (at  Thimble  Shoal)  vertical 
levels  at  the  measurement  locations.  Delft3D  has  4  vertical  levels  while  ADCIRC3D  has 
41  levels  at  all  three  NOAA/NOS  instrument  locations. 


Table  8 

RMSE  at  Naval  Station  for  Depth-Integrated  Currents 


ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

6/4-6/6 

0.177 

0.215 

0.213 

0.227 

6/6— 6/8 

0.174 

0.201 

0.211 

0.211 

6/8-6/10 

0.152 

0.194 

0.197 

0.206 

6/4-6/10 

0.168 

0.203 

0.207 

0.215 
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Figure  21.  The  Root  Mean  Squared  Error  (RMSE)  of  numerical  models  for  depth-integrated  currents 
predictions  between  June  4  and  10, 2010,  at  Cape  Henry,  Thimble  Shoal,  and  Naval  Station  locations. 
(Color  figure  available  online.) 


As  ADCIRC3D,  NCOM,  and  Delft3D  have  different  vertical  resolutions  over  the 
water  column  and  use  different  bathymetry  databases,  correlation  coefficients  and  RMSE 
are  computed  only  at  common  depths  where  all  three  models  and  ADP  data  have  current 
velocities.  The  model  results  cannot  be  compared  to  the  measurements  at  the  surface  since 
the  ADP  is  downward  looking  and  there  is  a  1.4  m  blanking  distance.  The  bottom  level  is 
also  not  used  since  the  depths  within  Delft3D  are  deeper  than  both  the  observations  and 
the  other  models  at  Thimble  Shoal.  As  a  result,  the  model-data  comparisons  are  done  at 
the  20%  and  60%  depth  levels  of  Delft3D.  The  NCOM  and  ADCIRC3D  results  have  been 
interpolated  to  make  the  comparisons  at  the  same  depths. 

Tables  9-11  show  the  correlation  coefficients  of  model  results  compared  to  the  ADP 
measurements  at  the  three  locations  during  June  4-10,  2010.  The  statistics  were  calculated 


Table  9 

Correlation  Coefficient  of  current  Profiles  at  Cape  Henry  at  0.2D  and  0.6D  depth 


Depth  Date 

ADCIRC3D 

NCOM 

Delft3D 

0.2D 

0.6D 

0.2D 

0.6D 

0.2D 

0.6D 

6/4— 6/6 

-0.223 

0.640 

0.499 

0.514 

0.187 

-0.235 

6/6— 6/8 

0.118 

0.773 

0.059 

0.414 

-0.126 

0.148 

6/8-6/10 

0.621 

0.408 

0.601 

0.694 

0.373 

0.164 
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Figure  22.  The  depth-varying  current  measurements  and  model  results  at  Cape  Henry,  Thimble 
Shoal  and  Naval  Station  NOAA/NOS  instrument  locations  at  0600  hr  on  June  4,  2010.  (Color  figure 
available  online.) 


Table  10 


Correlation  Coefficient  of  current  Profiles  at  Thimble  Shoal  at  0.2D  and  0.6D  depth 


Depth  Date 

ADCIRC3D 

NCOM 

Delft3D 

0.2D 

0.6D 

0.2D 

0.6D 

0.2D 

0.6D 

6/4-616 

-0.173 

0.160 

-0.125 

0.356 

-0.289 

-0.131 

6/6-618 

0.245 

0.262 

0.148 

0.542 

0.100 

-0.052 

6/8-6/10 

0.292 

-0.009 

0.130 

0.099 

0.279 

-0.186 

Table  11 

Correlation  Coefficient  of  current  Profiles  at  Naval  Station  at  0.2D  and  0.6D  depth 


Depth  Date 

ADCIRC3D 

NCOM 

Delft3D 

0.2D 

0.6D 

0.2D 

0.6D 

0.2D 

0.6D 

6/4— 6/6 

-0.037 

0.073 

-0.173 

0.137 

-0.016 

-0.177 

6/6— 6/8 

-0.111 

-0.075 

0.088 

-0.113 

0.165 

0.038 

6/8-6/10 

0.238 

0.169 

-0.138 

-0.188 

0.017 

0.145 
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Figure  23.  The  depth-varying  current  measurements  and  model  results  at  Cape  Henry,  Thimble 
Shoal  and  Naval  Station  NOAA/NOS  instrument  locations  at  0600  hr  on  June  5,  2010.  (Color  figure 
available  online.) 


at  the  20%  and  60%  Delft3D  depth  levels,  denoted  0.2D  and  0.6D,  respectively,  using 
hourly  measurements  and  hourly  model  results.  The  results  show  that  each  model  has  the 
highest  correlation  coefficient  at  least  once  at  one  of  the  stations  during  the  three  48-hr 
periods  considered.  ADCIRC3D  and  NCOM  perform  better  than  Delft3D  at  Cape  Henry 
and  Thimble  Shoal.  In  fact,  Delft3D  has  the  highest  correlation  only  once  and  at  the  Naval 
Station.  The  correlation  coefficients  are  higher  at  Cape  Henry  indicating  that  the  model 
phase  is  in  better  agreement  with  the  measurements  at  that  location.  Neither  ADCIRC3D 
nor  NCOM  is  consistently  better  than  the  other  at  Cape  Henry.  ADCIRC3D  predictions  are 
better  correlated  with  the  measurements  at  0.2D  level  at  Thimble  Shoal  while  NCOM  has 
the  highest  correlation  at  all  times  at  the  0.6D  level  at  the  same  location.  It  may  be  seen  that 
as  we  move  further  upstream  in  the  bay,  the  phase  errors  of  all  3  models  increase.  This  may 
be  attributed  to  the  increased  nonlinearities  at  those  shallower  locations  in  which  proper 
physics  are  not  incorporated  in  the  models. 

The  RMSEs  were  calculated  at  0.2D  and  0.6D  depth  levels  using  hourly  measurements 
and  hourly  model  data  and  are  presented  in  Tables  12-14.  NCOM  current  predictions  have 
the  least  error  at  0.2D  vertical  level  at  Cape  Henry  while  ADCIRC3D  has  the  least  error 
at  0.6D  for  the  first  2-day  periods.  Delft3D  generally  has  the  largest  error  in  predicting 
currents.  At  Thimble  Shoal,  NCOM  results  have  the  least  error  at  0.6D  while  ADCIRC3D 
and  Delft3D  produce  current  predictions  closer  to  the  measurements  at  0.2D.  Finally,  at 
Naval  Station  ADCIRC3D  results  show  the  least  error  at  0.2D  and  NCOM  results  have  the 
smallest  error  at  0.6D  while  Delft3D  produces  the  highest  error  at  both  depths.  Overall,  all 
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Figure  24.  The  depth-varying  current  measurements  and  model  results  at  Cape  Henry,  Thimble 
Shoal,  and  Naval  Station  NOAA/NOS  instrument  locations  at  0600  hr  on  June  6,  2010.  (Color  figure 
available  online.) 


Table  12 

Root-mean- square  error  of  current  Profiles  at  Cape  Henry  at  0.2D  and  0.6D  depth 


ADCIRC  3D  NCOM  Delft3D 


Depth  Date 

0.2D 

0.6D 

0.2D 

0.6D 

0.2D 

0.6D 

6/4-616 

0.271 

0.149 

0.187 

0.154 

0.236 

0.250 

6/6— 6/8 

0.331 

0.226 

0.316 

0.287 

0.348 

0.342 

6/8-6/10 

0.253 

0.254 

0.237 

0.195 

0.286 

0.299 

Table  13 

Root-mean-square  error  of  current  Profiles  at  Thimble  Shoal  at  0.2D  and  0.6D  depth 

ADCIRC3D 

NCOM 

Delft3D 

Depth  Date 

0.2D 

0.6D 

0.2D 

0.6D 

0.2D 

0.6D 

6/4— 6/6 

0.196 

0.250 

0.142 

0.164 

0.140 

0.201 

6/6— 6/8 

0.167 

0.239 

0.170 

0.182 

0.155 

0.227 

6/8-6/10 

0.192 

0.308 

0.199 

0.233 

0.216 

0.260 
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Figure  25.  The  depth-varying  current  measurements  and  model  results  at  Cape  Henry,  Thimble 
Shoal  and  Naval  Station  NOAA/NOS  instrument  locations  at  0600  hr  on  June  7,  2010.  (Color  figure 
available  online.) 


three  models  produce  similar  error  in  predicting  the  currents,  but  ADCIRC3D  and  NCOM 
produce  more  vertical  variability  and  hence  a  more  realistic  current  structure  over  the  water 
column.  Delft3D  has  only  4  vertical  levels,  and  this  may  be  one  of  the  limiting  factors 
leading  to  less  accurate  predictions. 


5.  System  Requirements  and  Operational  Related  Issues 

5.7.  Hardware  Requirements 

ADCIRC2D:  The  system  was  designed  to  be  independent  of  the  hardware  platform.  The 
Chesapeake  Bay  System  was  run  on  the  NRL  Linux  cluster  as  well  as  on  the  NAVO  DSRC 


Table  14 


Root-mean-square  error  of  current  Profiles  at  Naval  Station  at  0.2D  and  0.6D  depth 


Depth  Date 

ADCIRC3D 

NCOM 

Delft3D 

0.2D 

0.6D 

0.2D 

0.6D 

0.2D 

0.6D 

6/4— 6/6 

0.149 

0.202 

0.188 

0.181 

0.221 

0.219 

6/6— 6/8 

0.134 

0.221 

0.144 

0.198 

0.246 

0.238 

6/8-6/10 

0.186 

0.208 

0.223 

0.220 

0.289 

0.241 
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Figure  26.  The  depth-varying  current  measurements  and  model  results  at  Cape  Henry,  Thimble 
Shoal  and  Naval  Station  NOAA/NOS  instrument  locations  at  0600  hr  on  June  8,  2010.  (Color  figure 
available  online.) 


IBM-P6  platform.  Both  systems  were  running  in  real-time  during  the  exercise  period.  The 
Linux  version  used  64  processors  and  took  about  1-hr  wall  clock  time  to  run  72-hr  forecasts. 
The  DSRC  version  required  approximately  the  same  time  with  the  same  configuration. 

ADCIRC3D:  This  system  was  also  designed  to  be  independent  of  the  hardware  plat¬ 
form.  In  order  to  wait  for  the  completion  of  NCOM  output  for  the  initial  and  boundary 
conditions,  ADCIRC3D  was  run  in  delayed  mode  on  the  NRL  SGE  platform  during  the 
exercise  period.  Each  72-hr  simulation  with  64  CPUs  took  4  hrs  of  wall  clock  time.  Similar 
computation  time  could  be  expected  for  the  DSRC  platform  since  the  SGE  and  the  DSRC 
IBM-P6  are  comparable  with  regard  to  processor  speed. 

NCOM:  During  the  exercise,  NCOM  used  the  most  computational  resources:  5  hrs  of 
wall  clock  time  using  128  CPUs  and  produced  raw  output  file  sizes  of  22GB/day.  This  is 
likely  due  to:  (1)  a  large  geographic  domain  containing  more  than  one  million  grid  points, 
(2)  relative  high  spatial  resolution  at  500  m  and  small  time  step  during  integration,  (3) 
large  numbers  of  vertical  layers,  and  4)  the  NCODA  data  assimilation  procedure  added  an 
additional  hour  of  CPU  time. 

DELFT3D:  The  model  was  configured  to  run  on  a  single  CPU  PC,  with  a  configuration 
consisting  of  250,000  cells  with  4  vertical  layers.  A  72-hr  run  took  4-5  hrs  of  wall  clock 
time,  and  the  output  files  required  about  2GB  of  disk  space. 

In  additional  to  the  computational  requirements,  one  other  factor  to  be  considered  for 
operational  daily  forecasts  is  the  size  of  the  model  output.  For  example,  a  typical  NCOM 
output  in  compressed  format  takes  more  than  22GB  of  disk  space.  Archiving  and  purging 
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Table  15 


Summary  of  model  computation  resource  requirements 


ADCIRC2D 

ADCIRC3D 

NCOM 

Delft3D 

72-hr  run  on  Davinci 

1  hour  wall 

NA 

5  hour  wall 

NA 

clock  on  64 

clock  on  128 

CPUs 

CPUs 

72-hr  run  On  NRL 

1  hour  wall 

4  hour  wall 

NA 

NA 

Linux  cluster 

clock  on  64 

clock  on  64 

CPUs 

CPUs 

72-hr  run  on  PC 

NA 

NA 

NA 

5  hour  single 

processor 

CPU 

Output  file  size 

1GB 

5GB 

22GB 

2GB 

CPU  in  second 

0.3 

3.0 

1.5 

0.05 

/cell/day  run 

Ratio 

6 

60 

30 

1 

procedures  need  to  be  carefully  evaluated  to  prevent  disk  storage  issues.  This  is  especially 
true  for  local  workstations  with  limited  storage  capacity. 

The  computational  resources  required  to  run  a  typical  72-hr  forecast  for  the  Chesapeake 
Bay  region  for  each  of  the  four  models  based  on  the  present  configuration  are  summarized 
in  Table  15.  Using  Delft3D  as  a  benchmark,  the  ratio  of  CPU  per  cell  per  day  for  each 
model  was  estimated  at  the  end  of  Table  15. 


5.2.  Personnel  Requirements 

Personnel  resource  requirements  for  running  a  new  geographic  region  on  a  regular  basis 
are  evaluated  based  on  three  categories:  (1)  initial  training,  (2)  set-up  and  configuration  of  a 
new  area,  and  (3)  daily  monitoring  and  maintenance.  Some  of  those  requirements  for  each 
modeling  system  are  summarized  in  Table  16. 

Initial  Training:  All  systems  have  fairly  user-friendly  software  installation  scripts 
and  documentation  (user’s  guide  and  software  manual).  Two-day  training/tutorial  sessions 
should  cover  all  the  necessary  steps  in  setting  up  new  geographic  domain,  using  mesh 
generation  tools,  modifying  run  scripts,  and  operational  and  maintenance  issues. 

Setting  up  a  new  domain:  Both  NCOM  and  Delft3D  have  a  relative  straightforward 
procedure  in  setting  up  a  new  domain  since  rectangular  grid  can  be  generated  automatically 
once  the  user  specifies  the  latitude  and  longitude  of  the  four  corners  of  the  model  domain. 
On  the  other  hand,  due  to  the  nature  of  the  triangular  unstructured  mesh  system  used  by 
ADCIRC,  mesh  generation  cannot  be  fully  automated  at  present.  One  of  the  concerns  re¬ 
garding  the  finite  element  based  coastal  forecasting  system  is  the  time  and  effort  required  to 
setup  a  new  geographic  region  and  generate  a  mesh.  The  MeshGUI  software  was  developed 
to  create  mesh,  and  a  step-by-step  user  guide  describing  how  to  generate  the  mesh  from 
scratch  was  compiled  to  assist  the  end  users  (Blain  et  al.  2008).  Using  the  NRL  in-house 
developed  mesh  generation  GUI  tools,  users  are  able  to  generate  a  new  domain  mesh  file 
for  ADCIRC  within  an  hour. 
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Daily  monitoring  and  maintenance:  All  four  systems  employ  scripts  for  automated 
daily  operation  once  the  system  is  configured.  Daily  forecasts  are  fully  automated  requiring 
no  special  maintenance.  Minimal  monitoring  is  needed  to  restart  the  system  in  case  of  inter¬ 
ruption  due  to  (1)  missing  or  delayed  input  fields,  (2)  hardware  failure,  and  (3)  insufficient 
local  storage  space. 

6.  Summary  and  Conclusions 

Three  coastal  models — one  community  code-ADCIRC2D/3D,  one  proprietary  model- 
NCOM,  and  one  commercial  software-Delft3D — have  been  configured,  tested,  and  val¬ 
idated  for  the  lower  Chesapeake  Bay  region  during  a  Navy  exercise  in  June  2010.  Water 
level  predictions  are  compared  with  a  NOAA/NOS  water  level  gauge  at  the  Chesapeake  Bay 
Bridge  Tunnel  location  while  the  current  predictions  are  validated  with  ADP  measurement 
records  at  Cape  Henry,  Thimble  Shoal,  and  Naval  Station.  Standard  statistical  metrics  such 
as  correlation  coefficient  and  RMSE  are  computed.  Both  vertically  integrated  currents  and 
currents  at  various  vertical  water  depths  are  compared. 

The  validation  results  and  statistics  for  surface  elevation  and  vertically  integrated 
currents  show  ADCIRC2D  and  NCOM  yield  better  statistics,  in  terms  of  correlation  and 
RMSE,  than  the  other  two  models.  For  the  horizontal  currents  in  the  vertical  direction,  the 
ADCIRC3D  and  NCOM  showed  better  agreement  with  the  NOAA  ADP  measurements. 

All  three  models,  ADCIRC3D,  NCOM,  and  Delft3D,  produced  currents  that  were  not 
well  correlated  with  the  meteorological  observations.  This  raises  the  possibility  that  the 
meteorological  model  forcing  was  in  some  way  suboptimal.  A  closer  look  at  the  COAMPS, 
particularly  the  spatial  and  temporal  resolutions,  indicated  the  27  km  resolution  at  3 -hr 
interval  is  not  adequate  to  resolve  the  fast  passing  weather  system  during  the  exercise 
period.  An  improved  method  of  assimilating  real-time  meteorological  station  data  should 
be  investigated  to  improve  the  meteorological  forcing  input.  Blain  et  al.  (2012)  showed  that 
surface  wind  forcing  may  be  a  significant  source  of  error  for  forecasts  in  coastal  waters 
and  enhancing  the  spatial  and  temporal  resolution  of  wind  predictions  will  improve  ocean 
model’s  predictability  of  coastal  currents. 

Large  errors  in  current  magnitude  were  found  at  several  levels  over  the  water  column 
from  the  model-data  comparisons.  The  reasons  for  those  discrepancies  and  low  correla¬ 
tion  coefficient  values  are  likely  due  to  (1)  water  depth  mismatches  among  models  and 
measurement  location,  (2)  inadequate  spatial  and  temporal  resolutions  for  COAMPS  wind 
forcing,  or  (3)  insufficient  number  of  vertical  layers  for  Delft3D.  The  winds  are  one  of 
the  dominant  mechanisms  for  the  currents  and  model  predictions  may  be  improved  espe¬ 
cially  if  atmospheric  forcing  is  provided  at  a  higher  spatial  and  temporal  resolution  since 
several  strong  wind  events  occurred  over  the  Chesapeake  Bay  area  during  the  validation 
period. 

The  resource  requirements  for  each  modeling  system  have  also  been  evaluated.  This 
includes  benchmark  tests  on  grid  generation,  model  setup  and  configuration,  as  well  as 
hardware  and  operational  requirements.  ADCIRC2D  and  NCOM  are  configured  to  run 
automatically  in  real-time  at  the  Navy  DoD  Supercomputing  Resources  Center  (DSRC). 
ADCIRC3D  can  be  configured  to  run  automatically.  Delft3D  currently  runs  on  a  single 
processor  PC  or  Linux  platform  and  it  cannot  be  configured  to  run  at  the  DSRC  until  the 
parallel  version  has  been  implemented. 

In  summary,  water  levels  and  currents  predicted  by  ADCIRC  and  NCOM  models 
showed  better  agreement  than  that  of  Delft3D  when  compared  with  the  Chesapeake  Bay 
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field  data  during  the  Navy  exercise.  The  present  four  vertical  layer  configuration  in  Delft3D 
is  not  adequate  to  resolve  the  dynamics  in  the  water  column,  and  the  bathymetry  data  used 
in  the  morphological  grid  should  be  verified  with  NAVO  DBDB2  bathymetry  database  or 
field  survey  data.  All  models  would  benefit  from  higher  spatial  and  temporal  resolution 
meteorological  forcing. 
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